External validation of triage tools for adults with suspected COVID-19 in a middle-income setting: an observational cohort study.

BACKGROUND: Tools proposed to triage ED acuity in suspected COVID-19 were derived and validated in higher income settings during early waves of the pandemic. We estimated the accuracy of seven risk-stratification tools recommended to predict severe illness in the Western Cape, South Africa. METHODS: An observational cohort study using routinely collected data from EDs across the Western Cape, from 27 August 2020 to 11 March 2022, was conducted to assess the performance of the PRIEST (Pandemic Respiratory Infection Emergency System Triage) tool, NEWS2 (National Early Warning Score, version 2), TEWS (Triage Early Warning Score), the WHO algorithm, CRB-65, Quick COVID-19 Severity Index and PMEWS (Pandemic Medical Early Warning Score) in suspected COVID-19. The primary outcome was intubation or non-invasive ventilation, death or intensive care unit admission at 30 days. RESULTS: Of the 446 084 patients, 15 397 (3.45%, 95% CI 34% to 35.1%) experienced the primary outcome. Clinical decision-making for inpatient admission achieved a sensitivity of 0.77 (95% CI 0.76 to 0.78), specificity of 0.88 (95% CI 0.87 to 0.88) and the negative predictive value (NPV) of 0.99 (95% CI 0.99 to 0.99). NEWS2, PMEWS and PRIEST scores achieved good estimated discrimination (C-statistic 0.79 to 0.82) and identified patients at risk of adverse outcomes at recommended cut-offs with moderate sensitivity (>0.8) and specificity ranging from 0.41 to 0.64. Use of the tools at recommended thresholds would have more than doubled admissions, with only a 0.01% reduction in false negative triage. CONCLUSION: No risk score outperformed existing clinical decision-making in determining the need for inpatient admission based on prediction of the primary outcome in this setting. Use of the PRIEST score at a threshold of one point higher than the previously recommended best approximated existing clinical accuracy.

Prognostic accuracy of triage tools for adults with suspected COVID-19 in a middle-income setting

1482 Figure 2Performance of tools predicting composite primary outcome for the Omicron period[Figure omitted. See PDF] 1482 Table 1Triage tool diagnostic accuracy statistics (95% CI) for predicting any adverse outcome (entire study period)Tool N* C-statistic Threshold N (%) above threshold Sensitivity Specificity PPV NPV CRB-65 432,584 0.70 (0.70, 0.71) >0 102,964 (23.8%) 0.61 (0.61, 0.61) 0.78 (0.77, 0.78) 0.09 (0.09, 0.09) 0.98 (0.98, 0.98) NEWS2 433,101 0.80 (0.79, 0.80) >1 178835 (41.3%) 0.83 (0.83, 0.83) 0.6 (0.6,0.6) 0.07 (0.07–0.07) 0.99 (0.99, 0.99) PMEWS 438,810 0.79 (0.79, 0.79) >2 199,386 (45.4%) 0.85 (0.85, 0.85) 0.56 (0.56, 0.56) 0.06 (0.06, 0.07) 0.99 (0.99,0.99) PRIEST 438,880 0.82 (0.82, 0.82) >4 158,893 (36.2%) 0.83 (0.83, 0.83) 0.65 (0.65,0.66) 0.08 (0.08, 0.08) 0.99 (0.99, 0.99) WHO 437,850 0.71 (0.71, 0.72) >0 235,775 (53.8%) 0.82 (0.81, 0.82) 0.47 (0.47, 0.47) 0.05 (0.05, 0.05) 0.99 (0.99, 0.99) TEWS 432,612 0.72 (0.71, 0.72) >2 134,097 (31%) 0.62 (0.62, 0.62) 0.70 (0.70, 0.70) 0.07 (0.07, 0.07) 0.98 (0.98, 0.98) Quick COVID 446,088 0.70 (0.69, 0.70) >3 35,145 (7.9%) 0.33 (0.33, 0.33) 0.93 (0.93, 0.93) 0.14 (0.14, 0.14) 0.98 (0.98, 0.98) *Patients with <3 parameters were excluded from analysis when estimating performance 1482 Table 2Triage tool diagnostic accuracy statistics (95% CI) for predicting any adverse outcome (Omicron period)Tool N* C-statistic Threshold N (%) above threshold Sensitivity Specificity PPV NPV CRB-65 136,961 0.69 (0.68, 0.70) >0 31,373 (22.9%) 0.59 (0.59, 0.59) 0.78 (0.78, 0.78) 0.05 (0.05, 0.05) 0.99 (0.99, 0.99) NEWS2 137,125 0.77 (0.76, 0.78) >1 76,183 (55.6%) 0.87 (0.87, 0.87) 0.45 (0.45, 0.45) 0.03 (0.03, 0.03) 0.99 (0.99, 0.99) PMEWS 138,954 0.76 (0.75, 0.76) >2 59,876 (43.1%) 0.80 (0.80, 0.80) 0.58 (0.58, 0.58) 0.04 (0.04, 0.04) 0.99 (0.99, 0.99) PRIEST 158,893 0.78 (0.77, 0.79) >4 46,529 (33.5%) 0.75 (0.75, 0.75) 0.67 (0.67, 0.67) 0.04 (0.04, 0.04) 0.99 (0.99, 0.99) WHO 138,666 0.62 (0.61, 0.63) >0 72,599 (52.4%) 0.70 (0.70, 0.70) 0.48 (0.48, 0.48) 0.03 (0.03, 0.03) 0.99 (0.99, 0.99) TEWS 136,967 0.73 (0.72, 0.74) >2 39,509 (28.8%) 0.64 (0.64, 0.64) 0.72 (0.72, 0.72) 0.04 (0.04, 0.04) 0.99 (0.99, 0.99) Quick COVID 140520 0.61 (0.60, 0.63) >3 8,210 (6.4%) 0.17 (0.17, 0.17) 0.94 (0.94, 0.94) 0.06 (0.06, 0.06) 0.98 (0.98, 0.98) *Patients with <3 parameters were excluded from analysis when estimating performanceResults and ConclusionOf the 446,084 patients, 15,397 patients (3.45%, 95% CI:34% to 35.1%) experienced the primary outcome. Figure 1 presents the ROC curves for the triage tools for the total study period and figure 2 for the period of the Omicron wave. NEWS2, PMEWS, PRIEST tool and WHO algorithm identified patients at risk of adverse outcomes at recommended cut-offs with moderate sensitivity (>0.8) and specificity ranging from 0.47 (NEWS2) o 0.65 (PRIEST tool). The low prevalence of the primary outcome, especially in the Omicron period, meant use of these tools would have more than doubled admissions with only a small reduction in risk of false negative triage.Triage tools developed specifically in low- and middle-income settings may be needed to provide accurate risk prediction. Existing triage tools may need to be used at varying thresholds to reflect different baseline-line risks of adverse outcomes in different settings.

Accuracy of emergency medical service telephone triage of need for an ambulance response in suspected COVID-19: an observational cohort study.

OBJECTIVE: To assess accuracy of emergency medical service (EMS) telephone triage in identifying patients who need an EMS response and identify factors which affect triage accuracy. DESIGN: Observational cohort study. SETTING: Emergency telephone triage provided by Yorkshire Ambulance Service (YAS) National Health Service (NHS) Trust. PARTICIPANTS: 12 653 adults who contacted EMS telephone triage services provided by YAS between 2 April 2020 and 29 June 2020 assessed by COVID-19 telephone triage pathways were included. OUTCOME: Accuracy of call handler decision to dispatch an ambulance was assessed in terms of death or need for organ support at 30 days from first contact with the telephone triage service. RESULTS: Callers contacting EMS dispatch services had an 11.1% (1405/12 653) risk of death or needing organ support. In total, 2000/12 653 (16%) of callers did not receive an emergency response and they had a 70/2000 (3.5%) risk of death or organ support. Ambulances were dispatched to 4230 callers (33.4%) who were not conveyed to hospital and did not deteriorate. Multivariable modelling found variables of older age (1 year increase, OR: 1.05, 95% CI: 1.04 to 1.05) and presence of pre-existing respiratory disease (OR: 1.35, 95% CI: 1.13 to 1.60) to be predictors of false positive triage. CONCLUSION: Telephone triage can reduce ambulance responses but, with low specificity. A small but significant proportion of patients who do not receive an initial emergency response deteriorated. Research to improve accuracy of EMS telephone triage is needed and, due to limitations of routinely collected data, this is likely to require prospective data collection.

Accuracy of telephone triage for predicting adverse outcomes in suspected COVID-19: an observational cohort study.

OBJECTIVE: To assess accuracy of telephone triage in identifying need for emergency care among those with suspected COVID-19 infection and identify factors which affect triage accuracy. DESIGN: Observational cohort study. SETTING: Community telephone triage provided in the UK by Yorkshire Ambulance Service NHS Trust (YAS). PARTICIPANTS: 40 261 adults who contacted National Health Service (NHS) 111 telephone triage services provided by YAS between 18 March 2020 and 29 June 2020 with symptoms indicating COVID-19 infection were linked to Office for National Statistics death registrations and healthcare data collected by NHS Digital. OUTCOME: Accuracy of triage disposition was assessed in terms of death or need for organ support up to 30 days from first contact. RESULTS: Callers had a 3% (1200/40 261) risk of serious adverse outcomes (death or organ support). Telephone triage recommended self-care or non-urgent assessment for 60% (24 335/40 261), with a 1.3% (310/24 335) risk of adverse outcomes. Telephone triage had 74.2% sensitivity (95% CI: 71.6 to 76.6%) and 61.5% specificity (95% CI: 61% to 62%) for the primary outcome. Multivariable analysis suggested respiratory comorbidities may be overappreciated, and diabetes underappreciated as predictors of deterioration. Repeat contact with triage service appears to be an important under-recognised predictor of deterioration with 2 contacts (OR 1.77, 95% CI: 1.14 to 2.75) and 3 or more contacts (OR 4.02, 95% CI: 1.68 to 9.65) associated with false negative triage. CONCLUSION: Patients advised to self-care or receive non-urgent clinical assessment had a small but non-negligible risk of serious clinical deterioration. Repeat contact with telephone services needs recognition as an important predictor of subsequent adverse outcomes.

Accuracy of emergency (999) telephone triage for predicting adverse outcomes in suspected COVID-19: an observational cohort study

Aims/Objectives/BackgroundIn the first wave of the pandemic some ambulance services received three times their usual number of 999 calls. The increase was mostly due to calls from patients with respiratory symptoms. Call handlers must rapidly decide whether patients need an emergency face-to-face assessment or could access non-emergency services.We assess accuracy of emergency telephone triage in identifying patients with suspected COVID-19 infection who need an ambulance response and identify factors which affect triage accuracy.Methods/DesignAn observational cohort study of adults who contacted 999 emergency telephone services provided by Yorkshire Ambulance Service between the 18thMarch 2020 and 29th June 2020 with symptoms indicating possible COVID-19 infection was completed. Callers were linked to ONS death registrations and routine health care data collected by NHS Digital.The accuracy of triage outcome (ambulance dispatch versus telephone advice) was assessed for death or organ support 30 days from first contact. Multi-variable logistic regression was used to identify factors associated with risk of false negative or false positive triage.Results/ConclusionsOf the 12, 655 callers, 11.1% experienced the primary outcomes. An ambulance was dispatched to 84.2% of callers. The decision to dispatch an ambulance achieved 95% sensitivity (95% CI: 93.7 to 96.1%) and 17.2% specificity (95% 16.5% to 17.9%) for adverse outcomes. Where an ambulance was not dispatched, patients had a 3.5% (2.8 to 4.4%) of subsequent deterioration. Of patients that received an ambulance only 57% were subsequently conveyed to hospital. Multivariable logistic regression modelling found false negative assessment was associated with younger age and female sex and false positive assessment was associated with malignancy, immunosuppression, respiratory and cardiovascular comorbidities.Emergency telephone triage of patients with suspected COVID-19 achieved a high sensitivity to serious adverse outcomes. Further research is required to identify ways specificity of triage could be improved to reduce unnecessary ambulance dispatch.

Accuracy of telephone triage for predicting adverse outcomes in suspected COVID-19: an observational cohort study

798 Figure 1STROBE flow diagram of study population selection[Figure omitted. See PDF] 798 Table 1Performance of binary NHS 111 triage (ambulance or urgent assessment 4 hours or less) for composite outcome (death or organ support)Adverse outcome up to 30 days (3%, 2.8-3.2%) N=40, 261 Adverse Outcome No Adverse Outcome Ambulance/urgent assessment 890 15, 035 Sensitivity 74.2% (71.6- 76.6%) Positive Predictive Value 5.6% (5.2 - 6%) Self-care/non-urgent assessment 310 24, 025 Specificity 61.5% (61% - 62%) Negative Predictive Value 98.7% (98.6 - 98.9%) Results/Conclusions3% of the 40,261 callers experienced an adverse outcome. Self-care/non-urgent assessment was recommended for 60%, with a small but non-negligible (1.3%) risk of subsequent deterioration. Triage achieved 74.2% sensitivity (95% CI: 71.6 to 76.6%) and 61.5% specificity (61% to 62%) for the primary outcome. Multivariable analysis suggested some co-morbidities (e.g. respiratory disease) may be over-estimated, and others (e.g. diabetes) underestimated, as predictors of deterioration. Repeat contact with services appears to be an important under recognised predictor of adverse outcomes with 2 contacts (OR 1.77 95% CI: 1.14 to 2.75) and 3+ contacts (OR 4.02 95% CI: 1.68 to 9.65) associated with clinical deterioration when not provided with an ambulance/urgent clinical assessment.

Prognostic accuracy of triage tools for adults with suspected COVID-19 in a prehospital setting: an observational cohort study.

BACKGROUND: Tools proposed to triage patient acuity in COVID-19 infection have only been validated in hospital populations. We estimated the accuracy of five risk-stratification tools recommended to predict severe illness and compared accuracy to existing clinical decision making in a prehospital setting. METHODS: An observational cohort study using linked ambulance service data for patients attended by Emergency Medical Service (EMS) crews in the Yorkshire and Humber region of England between 26 March 2020 and 25 June 2020 was conducted to assess performance of the Pandemic Respiratory Infection Emergency System Triage (PRIEST) tool, National Early Warning Score (NEWS2), WHO algorithm, CRB-65 and Pandemic Medical Early Warning Score (PMEWS) in patients with suspected COVID-19 infection. The primary outcome was death or need for organ support. RESULTS: Of the 7549 patients in our cohort, 17.6% (95% CI 16.8% to 18.5%) experienced the primary outcome. The NEWS2 (National Early Warning Score, version 2), PMEWS, PRIEST tool and WHO algorithm identified patients at risk of adverse outcomes with a high sensitivity (>0.95) and specificity ranging from 0.3 (NEWS2) to 0.41 (PRIEST tool). The high sensitivity of NEWS2 and PMEWS was achieved by using lower thresholds than previously recommended. On index assessment, 65% of patients were transported to hospital and EMS decision to transfer patients achieved a sensitivity of 0.84 (95% CI 0.83 to 0.85) and specificity of 0.39 (95% CI 0.39 to 0.40). CONCLUSION: Use of NEWS2, PMEWS, PRIEST tool and WHO algorithm could improve sensitivity of EMS triage of patients with suspected COVID-19 infection. Use of the PRIEST tool would improve sensitivity of triage without increasing the number of patients conveyed to hospital.